Massive Text Embedding Benchmark (MTEB) Leaderboard. To submit, refer to the MTEB GitHub repository 🤗 Refer to the MTEB paper for details on metrics, tasks and models.

  • Total Datasets: 62
  • Total Languages: 112
  • Total Scores: >5550
  • Total Models: 74

Overall MTEB English leaderboard 🔮

  • Metric: Various, refer to task tabs
  • Languages: English, refer to task tabs for others
Rank
Model
Embedding Dimensions
Sequence Length
Average (56 datasets)
Classification Average (12 datasets)
Clustering Average (11 datasets)
Pair Classification Average (3 datasets)
Reranking Average (4 datasets)
Retrieval Average (15 datasets)
STS Average (10 datasets)
Summarization Average (1 dataset)
1
1024
512
62.25
75.24
44.49
86.03
56.61
50.56
82.05
30.19
2
768
512
61.79
73.12
44.74
86.62
57.29
49.26
83.06
32.32
3
768
512
61.59
73.86
45.29
85.89
57.54
47.57
83.15
31.84
4
768
512
61.5
73.84
43.8
85.73
55.91
50.29
81.05
30.28
5
1024
514
61.5
74.81
41.06
84.75
55.86
51.43
81.56
29.69
6
1024
512
61.42
73.14
43.33
85.94
56.53
49.99
82.06
30.97
7
1536
8191
60.99
70.93
45.9
84.89
56.32
49.25
80.97
30.8
8
768
512
60.44
72.63
42.11
85.09
55.7
48.75
80.96
31.01
9
384
512
59.93
72.94
39.92
84.67
54.32
49.04
80.39
31.16
10
768
512
59.54
72.36
41.9
83.51
56.2
45.12
82.29
29.85
11
768
512
59.51
73.42
43.72
85.06
56.42
42.24
82.63
30.08
12
768
514
59.45
73.02
37.89
83.57
54.84
48.88
80.26
30.11
13
768
512
58.97
67.41
42.42
86.12
56.66
48.48
78.38
30.64
14
4096
2048
58.93
68.13
40.34
82
56.56
50.25
78.1
31.46
15
384
512
58.89
71.67
39.51
85.08
54.45
46.01
80.87
31.39
16
768
512
58.42
67.11
41.51
86.13
55.96
47.96
77.8
30.21
17
768
512
58.28
67.14
41.6
85.32
55.36
47.42
78.19
29.5
18
384
512
57.87
70.74
37.08
82.59
53.87
46.64
79.1
29.98
19
768
512
57.87
72.84
42.34
86.06
54.71
38.47
81.66
29.91
20
768
514
57.78
65.07
43.69
83.04
59.36
43.81
80.28
27.49
21
4096
2048
57.59
66.19
38.93
81.9
55.65
48.22
77.74
33.6
22
2560
2048
57.17
67.13
39.83
80.65
54.67
46.54
76.83
31.03
23
768
512
57.06
72.31
41.65
84.97
54
36.71
81.83
29.64
24
384
512
56.53
63.21
41.81
82.41
58.44
42.69
79.8
27.9
25
384
512
56.26
63.05
42.35
82.37
58.04
41.95
78.9
30.81
26
2048
2048
56.2
66.52
39.92
79.58
54
44.49
75.74
30.43
27
768
512
56.19
65.25
38.63
83.85
54.23
44.67
77.07
29.67
28
768
512
56
66.68
41.1
82.54
53.14
41.88
76.51
30.36
29
768
512
55.27
69.81
40.21
85.18
53.09
33.63
81.14
31.39
30
768
514
54.71
67.9
38.4
80.81
53.8
35.34
80.73
31.57
31
4096
2048
53.74
70.14
36.98
77.03
52.33
32.34
80.53
30.38
32
384
512
52.44
64.3
37.14
78.45
53.62
32.45
78.93
30.67
33
768
512
52.35
64.71
37.64
81.74
51.84
32.96
76.47
29.5
34
768
2048
51.25
60.72
35.79
75.23
50.58
37.04
73.41
29.71
35
1024
2046
49.52
70.44
37.52
76.86
49.02
18.36
78.6
26.94
36
768
512
48.87
67.32
33.43
73.68
47.54
21.82
79.12
31.17
37
768
2048
45.97
61.46
30.95
71.78
47.56
20.9
74.71
30.26
38
768
512
45.45
62.5
29.04
70.33
46.47
20.29
74.33
31.15
39
768
512
45.21
62.71
29.55
78.87
48.42
18.99
70.8
31.05
40
300
N/A
42.06
57.65
26.57
72.94
44.75
21.22
62.46
30.49
41
300
N/A
41.96
57.29
27.73
70.92
43.29
21.62
61.85
28.87
42
768
512
40.28
52.37
34.06
61.37
48.1
15.88
61.02
27.66
43
768
512
38.33
61.66
30.12
56.33
43.44
10.59
54.36
29.82
44
1024
N/A
34.95
53.18
15.28
68.86
41.44
7.94
63.27
26.8
45
768
514
46
768
512
47
1024
512
48
768
512
49
1024
512
50
768
512
51
2048
2046
77.46
52
4096
2046
77.79
53
12288
2046
75.9
54
1024
2046
55
1024
2046
56
2048
2046
57
4096
2046
58
12288
2046
59
768
512
60
1024
514
61
768
62
70.78
40.58
82.15
58.88
79
30.15
63
65.45
43.8
83.04
59.36
80.28
27.49
64
65.78
35.06
79.62
75.35
29.71
65
63.42
34.82
75.43
75.39
30.79
66
64.45
35.71
76.23
72.04
29.42
67
67.9
37.82
79.53
74.05
29.01
68
64.85
74.53
75.11
69
30.62
70
71
384
512
62.23
30.61
76.49
80.24
30.1
72
1024
512
47.85
70.81
29.02
73
768
514
61.44
37.92
82.11
55.24
74
512
512
60.12
32.42
78.43
53
77.19
28.96

Made with ❤️ for NLP. If this work is useful to you, please consider citing:

@article{muennighoff2022mteb,
    doi = {10.48550/ARXIV.2210.07316},
    url = {https://arxiv.org/abs/2210.07316},
    author = {Muennighoff, Niklas and Tazi, Nouamane and Magne, Lo{\"\i}c and Reimers, Nils},
    title = {MTEB: Massive Text Embedding Benchmark},
    publisher = {arXiv},
    journal={arXiv preprint arXiv:2210.07316},  
    year = {2022}
}